Scientific workflow orchestration interoperating HTC and HPC resources

8 páginas, 7 figuras.-- El Pdf del artículo es la versión pre-print.In this work we describe our developments towards the provision of a unified access method to different types of computing infrastructures at the interop- eration level. For that, we have developed a middleware suite which bridges not interoperable middleware stacks used for building distributed computing infrastructues, UNICORE and gLite. Our solution allows to transparently access and operate on HPC and HTC resources from a single interface. Using Kepler as workflow manager, we provide users with the needed integration of codes to create scientific workflows accessing both types of infrastructures.Peer reviewe

Scientific workflow orchestration interoperating HTC and HPC resources

8 páginas, 7 figuras.-- El Pdf del artículo es la versión pre-print.In this work we describe our developments towards the provision of a unified access method to different types of computing infrastructures at the interop- eration level. For that, we have developed a middleware suite which bridges not interoperable middleware stacks used for building distributed computing infrastructues, UNICORE and gLite. Our solution allows to transparently access and operate on HPC and HTC resources from a single interface. Using Kepler as workflow manager, we provide users with the needed integration of codes to create scientific workflows accessing both types of infrastructures.Peer reviewe

INDIGO-DataCloud: A data and computing platform to facilitate seamless access to e-infrastructures

This paper describes the achievements of the H2020 project INDIGO-DataCloud. The project has provided e-infrastructures with tools, applications and cloud framework enhancements to manage the demanding requirements of scientific communities, either locally or through enhanced interfaces. The middleware developed allows to federate hybrid resources, to easily write, port and run scientific applications to the cloud. In particular, we have extended existing PaaS (Platform as a Service) solutions, allowing public and private e-infrastructures, including those provided by EGI, EUDAT, and Helix Nebula, to integrate their existing services and make them available through AAI services compliant with GEANT interfederation policies, thus guaranteeing transparency and trust in the provisioning of such services. Our middleware facilitates the execution of applications using containers on Cloud and Grid based infrastructures, as well as on HPC clusters. Our developments are freely downloadable as open source components, and are already being integrated into many scientific applications

EUROfusion Integrated Modelling (EU-IM) capabilities and selected physics applications

International audienceRecent developments and achievements of the EUROfusion Code Development for Integrated Modelling project (WPCD), which aim is to provide a validated integrated modelling suite for the simulation and prediction of complete plasma discharges in any tokamak, are presented. WPCD develops generic complex integrated simulations, workflows, for physics applications, using the standardized European Integrated Modelling (EU-IM) framework. Selected physics applications of EU-IM workflows are illustrated in this paper

Exploitation of e-infrastructures providing seamless access to e-science environments

[ES]: La computación avanzada se ha convertido en un factor crucial en la mayoría de las áreas de la ciencia, en algunos casos tan crucial como pueda ser la observación experimental. El análisis de los datos o la validación experimental de estos requiere por parte de los instrumentos de observación (detectores, sensores, etc...) la capacidad de comunicar e interaccionar con los recursos computacionales y herramientas de software capaces de almacenar y dar formato a los datos que los científicos analizan. Este entorno multidisciplinar y colaborativo es lo que se conoce como e-Ciencia. En esta memoria se presentan varias soluciones desarrolladas para facilitar el acceso transparente a recursos distribuidos que permitirán el acceso de científicos con una limitada formación específica. Los problemas que se han abordado tienen que ver con la interactividad en el acceso a recursos, y con el diseño de flujos de trabajo. De esta forma se pueden simular sistemas complejos integrados por diferentes elementos que interaccionan unos con otros. A lo largo de todo el trabajo se ha colaborado con investigadores del área de Fusión Nuclear y Astrofísica para implementar las soluciones en entornos de computación científicos reales para los investigadores. Se presentan varios ejemplos de flujos de trabajo complejos, que son prototipos de lo que podrá ser una plataforma de simulación del plasma de un reactor de fusión y análisis de datos para el experimento CMB/Planck. De esta forma hemos demostrado la versatilidad de las herramientas desarrolladas, al aplicarlas a más de una disciplina científica.[EN]: Advanced computing has become a crucial factor in most areas of science , in some cases may be as critical as the experimental observation . The data analysis and experimental validation of these needs by observation instruments (detectors , sensors, etc ... ) the ability to communicate and interact with computing resources and software tools capable of storing and formatting the scientists analyzed data . This multidisciplinary and collaborative environment is what is known as e- Science . In this work, several solutions have been developed to facilitate transparent access to distributed resources that allow scientists to access a limited specific training are presented herein . The problems that have been addressed have to do with interactivity in access to resources , and the design of workflows. This will be made up of various elements simulate complex systems that interact with each other. Throughout all the work we have collaborated with researchers in Nuclear Fusion and Astrophysics to implement solutions in real scientific computing environments for researchers. Several examples of complex workflows , which are prototypes of what will be a platform for simulation of plasma from a fusion reactor and analysis for the WBC / Planck experiment are presented . Thus we have demonstrated the versatility of the developed tools , when applied to more than one scientific discipline.I would like to thank also all the collaborators from projects that I was participating during my research (including int.eu.grid, EUFORIA and EGI InSPIRE), for very fruitful cooperation. The work presented in this thesis was partially supported by research grants from the FP7 (FP7/2007-2013) under grant agreement no211804 (EUFORIA), EGI InSPIRE project, under grant agreement no261323, EU 6FP Interactive European Grid project (2006-2008) contract no. 031857 .Peer Reviewe

Exploitation of e-infrastructures providing seamless access to e-science environments

RESUMEN: La computación avanzada se ha convertido en un factor crucial en la mayoría de las áreas de la ciencia, en algunos casos tan crucial como pueda ser la observación experimental. El análisis de los datos o la validación experimental de estos requiere por parte de los instrumentos de observación (detectores, sensores, etc...) la capacidad de comunicar e interaccionar con los recursos computacionales y herramientas de software capaces de almacenar y dar formato a los datos que los científicos analizan. Este entorno multidisciplinar y colaborativo es lo que se conoce como e-Ciencia. En esta memoria se presentan varias soluciones desarrolladas para facilitar el acceso transparente a recursos distribuidos que permitirán el acceso de científicos con una limitada formación específica. Los problemas que se han abordado tienen que ver con la interactividad en el acceso a recursos, y con el diseño de flujos de trabajo. De esta forma se pueden simular sistemas complejos integrados por diferentes elementos que interaccionan unos con otros. A lo largo de todo el trabajo se ha colaborado con investigadores del área de Fusión Nuclear y Astrofísica para implementar las soluciones en entornos de computación científicos reales para los investigadores. Se presentan varios ejemplos de flujos de trabajo complejos, que son prototipos de lo que podrá ser una plataforma de simulación del plasma de un reactor de fusión y análisis de datos para el experimento CMB/Planck. De esta forma hemos demostrado la versatilidad de las herramientas desarrolladas, al aplicarlas a más de una disciplina científica.ABSTRACT: Advanced computing has become a crucial factor in most areas of science , in some cases may be as critical as the experimental observation . The data analysis and experimental validation of these needs by observation instruments (detectors , sensors, etc ... ) the ability to communicate and interact with computing resources and software tools capable of storing and formatting the scientists analyzed data . This multidisciplinary and collaborative environment is what is known as e- Science . In this work, several solutions have been developed to facilitate transparent access to distributed resources that allow scientists to access a limited specific training are presented herein . The problems that have been addressed have to do with interactivity in access to resources , and the design of workflows. This will be made up of various elements simulate complex systems that interact with each other. Throughout all the work we have collaborated with researchers in Nuclear Fusion and Astrophysics to implement solutions in real scientific computing environments for researchers. Several examples of complex workflows , which are prototypes of what will be a platform for simulation of plasma from a fusion reactor and analysis for the WBC / Planck experiment are presented . Thus we have demonstrated the versatility of the developed tools , when applied to more than one scientific discipline

EOSC-SYNERGY. EU DELIVERABLE: D1.1 Exploitation and Dissemination Plan

Description of the activities and procedures to be executed in order to maximise the impact of the project and to protect the results, establishing the roadmap for their future exploitation.EOSC-SYNERGY receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 857647.Peer reviewe

EOSC SYNERGY Tools Development. Hackathon as a Service

[ES] El 23 de junio FAIRsFAIR en colaboración con la Unidad de Recursos de Información Científica para la Investigación (URICI) del Consejo Superior de Investigaciones Científicas (CSIC) organizan un evento online abierto a toda la comunidad española sobre la Nube Europea de Ciencia Abierta (EOSC) y la participación de los repositorios. El evento tiene como título "Green Data Market for Research & the Role of Repositories" y abordará, a través de intervenciones de integrantes del proyecto FAIRsFAIR, así como de diferentes iniciativas españolas y europeas, aspectos relativos a buenas prácticas en las políticas y gestión de repositorios de datos de investigación para su adhesión a los Principios FAIR y su inclusión en EOSC. El evento tendrá un formato bilingue y dará una panorámica de recomendaciones, servicios, recursos y formación para los repositorios de datos a través de experiencias del propio proyecto FAIRsFAIR como del Ministerio de Ciencia e Innovación de España, Science Europe, EOSC SYNERGY, IBERGRID y el CSIC. [EN] On June 23, FAIRsFAIR in collaboration with the Scientific Information Resources Unit for Research (URICI) of the Higher Council for Scientific Research (CSIC) organizes an online event open to the entire Spanish community on the European Open Science Cloud (EOSC) and the participation of data repositories. The event is entitled "Green Data Market for Research & the Role of Repositories" and will address, through interventions by members of the FAIRsFAIR project, as well as different Spanish and European initiatives, aspects related to good practices in the policies and management of research data repositories for adherence to the FAIR Principles and inclusion in EOSC. The event will have a bilingual format and will provide an overview of recommendations, services, resources and training for data repositories through experiences from the FAIRsFAIR project itself as well as from the Ministry of Science and Innovation of Spain, Science Europe, EOSC SYNERGY, IBERGRID and the CSIC.N

EOSC-SYNERGY. EU DELIVERABLE D4.2 - First prototype of the EOSC Thematic services (Demonstration)

This document describes the thematic services that expose the applications and data to the scientific community. This is a report that supports the demonstration deliverable D4.2. The 10 thematic services of EOSC-SYNERGY are reasonably different in maturity, requirements, technological needs and approach, which guarantees that the expansion of the EOSC capacity addresses multiple dimensions and challenges. The document describes four of them more in detail, as planned in the DoA.EOSC-SYNERGY receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 857647.Peer reviewe

EOSC-SYNERGY. EU DELIVERABLE: D2.1 Roadmap for integration of national capacities into the EOSC and Policy Gap analysis

This deliverable summarises the considerations made and decisions taken to arrive at the initial roadmap for the integration of national capacities. To provide thematic services with available infrastructure as early as possible, the majority of the resources will be integrated with high priority with the EGI Federated Cloud, while the more work intensive components will be integrated after the “low hanging fruit” have already been integrated. The current status of adoption of the AARC Policy Pack across the infrastructure was assessed with a questionnaire, results of which are provided. Room for improvement was identified and will be addressed with targeted online training on identified gaps. As a result of this deliverable, the majority of the infrastructure can be integrated with EOSC via the EGI Federated Cloud, while a smaller part of the capacities requires a case by case handling and adaptation / integration of software solutions into the operation of the participating infrastructure partners.EOSC-SYNERGY receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 857647.Peer reviewe